Rendering/ Animation with Global Illumination

Animation with Global Illumination

 

Computing the global illumination by random sampling light ray bounces is somewhat problematic for animations. Even when using a very high amount of light samples per pixel and heavy filtering, the computed illumination includes some random noise. Human eye easily notices even a low amount of noise. Just a couple of percents of noise from the full color intensity is still clearly visible. The constantly changing noise pattern usually appears quite unpleasant and distracting.

In this tutorial, we will examine how the noise problem can be solved using multi pass rendering. The big idea is to extend the illumination filtering over multiple frames. Also other random sampled shading components will be filtered in order to improve the quality.


An example shot from the final animation

To skip the details about how to configure multi pass rendering, go to the last section Multi Pass Rendering Summary. The summary tells you how to use the predefined components, which come with the installation.

Tutorial level: Advanced

Example projects: tutorprojects/rendering/gi_room.r3d, gi_post1.r3d, gi_post2.r3d

Contents:



The First Pass: Ray Tracing Deep Files

1. Model a simple test scene or load the provided test scene 'tutorprojects/rendering/gi_room.r3d'. The test scene includes one area light source in the ceiling. Unless a high sampling rate and slow rendering is used, the shadows appear badly dithered. Nevertheless, you can set the area light's sampling rate to a low value, so that rendering time will be appropriate for animation use. If you model the test scene yourself, animate the scene for example by moving the camera along a path.


The test scene. The area light source is sampled with the minimal quality. Shadows are badly dithered.

2. We will apply global illumination to the scene. Select the root level of the scene hierarchy. Go to the materials tab of the select window. If the standard material GI_shader is not yet included in the material list, select New/From Template from the popup menu and load in the file 'materials/illumination/GI_shader.r3d'. Then drag and drop the material to the view window to apply the material to the whole scene.

3. We will also filter the dithered shadows and other bad quality shading components. An easy way to do this is to use the provided example material 'materials/illumination/capture_channels.r3d'. Load it in using the New/From Template item of the popup menu. Then drag and drop the loaded material to the view window. The channel capture material gets assigned to the root level, too.


The hierarchy of the scene

4. At this point, you can switch to the post effects tab of the select window and make sure that GI_effects and GI_draft post processing configurations are available. If not, apply the popup menu's Paste from a file tool to insert the mentioned effects from the realsoft3d/posteffects folder. Load in also the configurations 'posteffects/post_shading.r3d' and 'post_shading_draft.r3d'. These are the example systems for filtering shadows, reflections etc. - see the tutorial Post Processing/Post Shading for details.

5. Open the view property window, go to the rendering tab and set GI_draft as the Effect/Image for the view window. Clear the Effect/Box selection. Change also the Render Settings object of the view window to the standard GI rendering choice. Go to the render settings tab, select GI rendering, open the property window and set Antialiasing level to zero. Antialiasing usually causes some troubles for post shading. Then apply Render/Ray Trace+Backup from the view popup menu. Adjust GI blurring and brightness parameters if necessary and apply view popup/Render/Post Process menu until the GI parameters seem to be suitable. This is just a preliminary test stage - you can change all post processing parameters later. That is one of the main points of this tutorial.

6. Use the view property window to change the view's Effect/Image setting to post_shading_draft. Then apply Ray Trace+ Backup, adjust post shading components and Post Process until the shadows appear non-dithered. You can change other post shading parameters, too, as instructed in the post shading tutorial. The purpose is to find preliminary parameter levels for the first rendering attempt.


Post shading applied to filter shadows and reflections

So far, the workflow has been pretty similar to any project, which uses global illumination (or post effects in general). Next we prepare the first render pass. Instead of applying the post effects directly, we will collect all untouched raw data into very deep image files. This gives us a possibility to adjust the post effects later to produce variations from the same raw data. If the final animation shows some insufficient filtering quality, or if the customer suddenly wants a warmer lighting into the scene, this approach saves the most time consuming ray tracing phase.

7. To define the deep image file contents, go to the output objects tab of the select window. Select New/Realsoft 3D from the popup menu. Make sure that the new format object is selected and open the property window. Use the name field to rename the format object for example as 'GI_BT_raw' (BT refers to the channel set which stores the post shading information).

We will have to add a large set of channels into this file format object. Select Distance from the Available Channels list and click Add. A new item labeled as Distance appears to the Image Channels box. Click the new image channel item and then change its Storage Type from Float to Word or Integer. Very deep files consume huge amount of hard disk space. By optimizing the storage data type, we can reduce the file size. Word is usually enough to store distances in a scene which has a diameter of some meters, Integer is sufficient for several kilometers and Float is enough for astronomical scales. After adding the distance, add other required channels - the full list is given below. Order of the channels does not matter.

  • Distance - Word
  • Bump Normal - Byte. The surface normal vector has unit length, therefore Byte is enough.
  • diffuse - Word. Diffuse illumination often has higher level than 1.0, therefore use Word.
  • unshaded - Byte. The surface color value for GI usually fits into a Byte.
  • spec_illum - Word. GI from reflections often needs higher bandwidth. Word is a good choice.
  • bt_reflection - Byte. Reflected illumination. Byte is enough for this scene.
  • bt_transparency - Byte. Colors seen trough glass. We can use Byte here.
  • bt_ambient - Byte. Self illumination in this scene fits into Byte range.
  • bt_specularity - Byte. Specular highlights can exceed Byte range, but truncating to level 1.0 (white) does not harm in this project, because we will not utilize high intensity values for special effects.
  • bt_diffuse_color - Byte. If you apply surface colors brighter than 1.0, use Word.
  • bt_diffuse_shading - Word. Diffuse Illumination from light sources needs some bandwidth.


Configuring the deep file format

In the list above, the data types are set to the minimal level in order to reduce the file size. If you have lots of free disk space available and feel uncertain about the right choice, just use one step better alternatives. Just remember that Float takes 8 times more space than Byte. The configuration below already consumes over 40 bytes per pixel. In the double PAL resolution, this means over 60 megabytes per image! A CD disk can only store about ten of such images. For a 250 frame animation, you will need 20 GB of empty hard disk space. However, some of the channels are almost empty and contain only some local compact peaks, such as specular highlights. Therefore, the deep image files usually compress very well and a backup of the 20 BG raw data can most likely be stored to a DVD disk by using .zip compression.

The illumination components diffuse, spec_illum and bt_diffuse shading are stored using Word data type. It may be possible to store them using Byte, if the lighting of the scene does not contain high intensity peaks. Insufficient data type increases sampling noise and therefore decreases rendering quality. If you use high intensity illumination effects, for example HDRI, bandwidth of the Byte data type is probably not enough.

Note: You can avoid the laborious file configuration step by saving predefined file format objects into a suitable folder. Realsoft3D/imageformats/GI_BT_raw.r3d contains the required .r3i format object for this tutorial. Use Paste from a File popup menu to load it into the output objects library. The example object uses larger data types, for example Integer instead of Word the Distance channel. You can check and optimize the data type for each channel if you like.

Note 2: The file format defined above does not include all post shading channels. Illumination, bt_total and bt_rn are not stored, because the example scene does not include fog and there is no flashlight included. The example version GI_BT_raw, which can be found from the imageformats folder, is more extensive and includes all post shading channels. You can delete the 3 additional channels to save some disk space. If you need these additional channels for other scenes, the recommended data types are:

  • Illumination - Word. The total result of shading. If you need to spare disk space and don't need to trace illumination peaks accurately, use Byte.
  • bt_rn - Byte. Byte is always enough for the flashlight coefficient.
  • bt_total - Word. Illumination before a possible global fog has been evaluated. Use the same data type as for the Illumination channel.

When rendering your own projects, you may have to expand the channel collection even more, depending on the applied post effects. For example, the glow effect needs glow size and glow color channels.

8. When the deep file format is ready, open the file rendering window (Ctrl+r). Add the deep file format, which we configured above, to the Active File Formats list and remove other formats. Give a suitable name and path for the image files (for example 'raytraced' because these files will hold purely raytraced information without any post effects). Select GI rendering as the applied Render Settings object. Clear all post effect selections. Finally, set the resolution to the double of the intended final resolution. If you plan to create a 480*360 pixel animation, double it to 960*720 pixels. The extra resolution will be used for antialiasing purposes. Save the project as pass1.r3d and then start rendering.


Rendering setup for the first pass


The Second Pass: Post Shading and GI Blurring Single Frames

With the second rendering pass, we will filter the noisy area lighting and the global illumination. GI, which is the noisiest and most problematic shading component, will not be added to the shading yet. It will be stored for another, multi-frame filtering pass.

9. Go to the geometric objects tab of the select window. We do not need the geometry in the second rendering pass, so you can either delete all objects or make them invisible in ray trace rendering. However, it is a good idea to leave the camera object intact, because some post effects (DOF, lens flares) depend on the camera. Although we do not use such effects in this tutorial, this is a safe working habit to learn.

10. We can optimize the render settings a bit. Go to the render settings tab, duplicate the GI rendering object and rename the copy as 'QuickRT' or something similar. Using the property window's Ray Tracing tab, set Horizontal and Vertical Undersampling to 4*4 pixels and turn Interpolation off. Make sure that Antialiasing Level is set to zero. Switch to the Distr tab and turn box rendering off. In the Post Proc tab, turn the Safety area option off ( an additional continuity area around the image would not improve anything, because the rendering data comes from the deep files, which do not cover the extension region). This kind of render setting configuration initializes the empty black image in a minimal time. You can also load in the optimized settings from the 'realsoft3d/rendersettings/QuickRT' file.

Note: the render settings object above uses only one single rendering thread. Because of filtering continuity issues, use of several threads would not speed up rendering much. Later, when using frame blur effects, hard disk IO for several rendering threads reading simultaneously the same, very large data files might actually slow down the rendering.

11. Go to the Image Effects tab. We will now configure a post effect system, which filters the shading information from the deep ray traced files. If you want to skip the details presented in the steps 11 - 14, load in the file 'realsoft3d/posteffects/GIPostShadeFiles.r3d'.

Because the filtering system will include the post shading components, one of the existing objects is a good starting point for the new system. So, select post_shading_draft and apply Duplicate from the popup menu. Select the new copy and open the property window. Rename the object as GIPostShadeFiles. The object already includes the post shading items, but GI blurring is not included. Therefore, drag and drop GI_blur and GI_blur_specular from the Available Objects list to the end of Active Effects list. We will not add GI_brightness_control, because the final GI composition is done during the third pass.

You can also optimize the post effect system slightly to gain some rendering speed:

  • Select 'remove_fog_and_custom_shading' from the Active Effects list and Apply Remove from the popup menu. There is no fog in this scene.
  • Select the 'blur_fogginess' from the Active Effects list and remove it, too.
  • On the select window, click the VSL effect evaluate_shading. On the property window, put the effect to Advanced mode and delete the following VSL objects from the shader tree:
    • Color fc - unused variable
    • Color+=Linear(bt_total) - no need for fog component
    • fc=multiply(bt_diffuse_color, bt_rn) - no flashligt needed
    • Color+=Linear(fc) - no flashligt needed

It does not harm if you skip these optimizations. Shading operations will be carried out using zero data. They do not change the image in any way; they just take some extra seconds of rendering time.


Removing the inactive post shading components speeds up rendering

Then some furthers optimizations: the two new GI blurring items, which we added above, need some channels, which were not referred by the original post shading system. Therefore, switch to the Channels tab of the property window. Find the channel diffuse from the list, click it and change the Storage to Word. Change also the spec_illum channel to Word. Now the new post effect configuration uses the same data types as the deep file format, to which we stored the rendering data. Storage types could be different, but we naturally want to use full bandwidth and accuracy of the stored channels.

12. One important effect is still missing: the one, which loads the processed channels from the deep files. Select New\VSL Effect from the popup menu. Rename the new effect as LoadPass1. Check the Advanced button of the property window. Activate the root node Image Processing and select New/Texture from the popup menu. Change the Input 0 parameter to UV coords, which is the default parameter for pixel processing. Then use the browse button to select the first image of the deep image sequence. Switch to the Animation tab of the property window and change the Animation Type to Automatic. Open the popup menu of the VSL tree and change the Output parameter to first one of the required channels, Image:Distance.

13. Duplicate the first Texture object of the VSL tree using the popup menu. Change the output parameter to the next channel, Image:Bump normal. Repeat this for the remaining channels: diffuse, unshaded, spec_illum, bt_reflection, bt_transparency, bt_ambient, bt_specularity, bt_diffuse_color and bt_diffuse_shading. Reading order does not matter.


The VSL effect, which reads channels

14. Select again the effect GIPostShadeFiles. Drag and drop the new VSL effect, which we just created, to the beginning of the effect list under the Image Processing shader. The filtering configuration is now ready.


The post effect system for the second rendering pass

15. Go to the output objects tab of the select window. Select New/Realsoft3D from the popup menu to add a new native format object (you can also load imageformats/GI_raw file to skip this step). Rename the new object as 'GI_Raw'. Then add the following channels to the object the same way as instructed in the step 7 above:

  • Color - Byte
  • Distance - Word
  • Bump Normal - Byte
  • diffuse - Word
  • unshaded - Byte
  • spec_illum - Word


The file format which stores the channels for the final pass

The new format no longer includes the 'bt_xxx' channel set of the post shading system - it is already composited into the Color channel. The other channels store the GI information for the final filtering pass.

16. Open the file rendering window. Change the file name and path so that images from the first pass won't get accidentally overwritten. Change the active file format to the new GI_raw object. Set the Render Settings selection to the QuickRT object, and Effect/Box to GIPostShadeFiles. The resolution remains the same (=the final resolution doubled). Save the project as 'pass2' and start rendering the animation.


Rendering the second pass.


The Third Pass: Multi Frame Filtering

17. When the second pass rendering is finished, go to the post effects tab of the select window. We will now configure the last filtering system. An example system is included in the file 'realsoft3d/posteffects/MultiFrameGI.r3d' - load it in to skip the steps 18-21 of the tutorial.

18. Select New/FrameBlur from the popup menu. Rename the created effect as FrameBlurDiffuse, because we will blur diffuse global illumination with it. Use the browse gadget to select an image from the second render pass. Switch to the Animation tab of the property window and change Animation Type to Automatic. Go back to the Image tab and set Blur Start and Blur End to 5 frames. This means that the effect will combine GI information from 5 frames before the current frame to 5 frames after the current frame, 11 frames total. In each frame, we will get ten times more information about the global illumination; this will help to reduce the noise significantly. Set Source and Destination Channels to diffuse. This means that the effect will read diffuse channel from the disk files and store the blurred result to the diffuse channel of the post processed image. Then set Geometric Weighting to 1.0 - frame blur should combine GI information only from geometrically continuous areas.

19. Duplicate the created FrameBlur effect. Rename the new effect as FrameBlurSpecular. This time set the source and destination channels to spec_illum. This part of the GI is usually much more coherent as the diffuse GI. Therefore, set Start and End to only 1 frames.


Frame blur effects for the diffuse and specular GI

20. Create a new VSL effect and rename it as LoadPass2. Set its Coordinate channel to UV coords and add four Texture objects, which read Color, unshaded, Bump normal and Distance from the pass 2 files. Remember to set the animation type of each Texture object to Automatic. Note that we do not have to read diffuse or spec_illum channels here - the two frame blur effects take care of that.


The VSL effect, which loads channels from the second pass images

21. Duplicate the post effect object GI_effects (use Paste from file popup menu to load it in from the posteffects folder, if it is not available yet). Open the property window. Rename the duplicate as MultiFrameGI. Add FrameBlurDiffuse above the GI_blur effect in the Active Effects list. Add FrameBlurSpecular after the diffuse frame blur. Then add the LoadPass2 effect after the two inserted efects. After that, there will be the usual list of GI components. Note that we will leave GI_blur and GI_blur_specular into the configuration. So, we first blur GI over image area, then over time and finally again within each frame. The purpose is to eliminate illumination banding artifacts, which easily occur, when a moving camera reveals previously obscured areas of the scene.

Also the usual post effects, such as glow, render particles and depth of field can be left into the configuration. They are inactive in this project. Most of these effects would not interfere with GI; however, depth of field, when active, should naturally take GI into account. Therefore, the correct place for DOF is after GI processing.


The post effect configuration for the final pass

22. Open the file rendering window. Now set the Active Outputs to a suitable 'normal' image or animation format, such as .jpg or .avi. Define also a suitable file name. The render settings should be again QuickRT. Set Post effect/Box to the new MultiFrameGI. Drop the resolution to the actual final resolution (e.g. 480*360) from the doubled value. MultiFrameGI, which has Output scaling factors 0.5, halves the resolution from its source data to antialiase the images. Save the project as pass3 and start rendering.


The final rendering pass


Multi Pass Rendering Summary

A brief summary of the multi pass rendering steps using the predefined components is given below. You can naturally use your own components instead of the objects, which come with the standard installation.

1. Model the scene as usual

2. Default map the materials 'realsoft3d/materials/illumination/GI_shader' and 'capture_channels' to the root level.

3. Render using the deep file format 'realsoft3d/imageformats/GI_BT_raw' in doubled resolution and with no antialiasing.

4. Load in the post effect system 'realsoft3d/posteffects/PostShadeFromFile'. Select the included VSL effect 'LoadPass1' and using the property window, change the file name of every Texture object to the image sequence you rendered in the previous pass.

5. Make all geometry objects except the camera RT invisible and render the second pass to 'realsoft3d/imageformats/GI_raw' format with an optimized render settings object 'realsoft3d/rendersettings/QuickRT' and PostShadeFromFile as the post effect per render box.

6. Load in the post effect system 'realsoft3d/posteffects/MultiFrameFiltering'. Update the image file names of the two FrameBlur effects to match the image files from the second pass. Update the file names of the VSL Texture objects in the VSL effect 'LoadPass2', too.

7. Render the final pass to the desired format with the QuickRT settings and the MultiFrameFiltering post effects.